Selecting speech features for building models  for detecting medical conditions

ABSTRACT

A mathematical model may be trained to diagnose a medical condition of a person by processing acoustic features and language features of speech of the person. The performance of the mathematical model may be improved by appropriately selecting the features to be used with the mathematical model. Features may be selected by computing a feature selection score for each acoustic feature and each language feature, and then selecting features using the scores, such as by selecting features with the highest scores. In some implementations, stability determinations may be computed for each feature and features may be selected using both the feature selection scores and the stability determinations. A mathematical model may then be trained using the selected features and deployed. In some implementations, prompts may be selected using computed prompt selection scores, and the deployed mathematical model may be used with the selected prompts.

CLAIM OF PRIORITY

This patent application is a continuation of U.S. patent application SerNo. 15/973,498 entitled “SELECTING SPEECH FEATURES FOR BUILDING MODELSFOR DETECTING MEDICAL CONDITIONS” and filed on May 7, 2018 for JangwonKim, et al., which claims the benefit of U.S. Provisional PatentApplication No. 62/502,584, filed May 5, 2017 and U.S. ProvisionalPatent Application No. 62/614,192, filed Jan. 5, 2018, all of which areincorporated herein by reference in their entireties for all purposes.

FIELD OF THE INVENTION

The present invention relates to selecting speech features to be usedfor building mathematical models for detecting medical conditions toimprove the performance of the models.

BACKGROUND

Early diagnosis of medical conditions, such as Alzheimer's disease orconcussions, may allow for improved treatment and improved quality oflife for the person with the medical condition. One method that may beused for detecting medical conditions is to process the speech of aperson because the sound of a person's voice or the words used by aperson may provide useful information for making a medical diagnosis.

To detect a medical condition from a person's speech, features may beextracted from the speech, and the features may be processed with amathematical model. The type and number of features extracted from thespeech may impact the performance of the model, especially where theamount of training data for training the model is limited. Accordingly,appropriate selection of features may improve the performance of themodel.

BRIEF DESCRIPTION OF THE FIGURES

The invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures:

FIG. 1 is an exemplary system for providing a service for diagnosing amedical condition by processing speech of a person.

FIG. 2 is an exemplary system for processing speech data with amathematical model to perform a medical diagnosis.

FIG. 3 is an example training corpus of speech data.

FIG. 4 is an example list of prompts for using in diagnosing a medicalcondition.

FIG. 5 is an exemplary system for selecting features for training amathematical model for diagnosing a medical condition.

FIGS. 6A and 6B are conceptual graphs of pairs of feature values anddiagnosis values.

FIG. 7 is a flowchart of an example implementation of selecting featuresfor training a mathematical model for diagnosing a medical condition.

FIG. 8 is a flowchart of an example implementation of selecting promptsfor use with a mathematical model for diagnosing a medical condition.

FIG. 9 is a flowchart of an example implementation of training amathematical model for diagnosing a medical condition that is adapted toa set of selected prompts.

FIG. 10 is an exemplary computing device that may be used to train anddeploy a mathematical model for diagnosing a medical condition.

DETAILED DESCRIPTION

Described herein are techniques for selecting features of speech to beused to build or train a mathematical model for detecting or diagnosinga medical condition. Although the techniques described herein may beused for any appropriate medical condition, for clarity of presentation,concussions and Alzheimer's disease will be used as examples of medicalconditions. The techniques described herein, however, are not limited toany particular medical conditions.

FIG. 1 is an example system 100 for diagnosing a medical condition usinga person's speech. FIG. 1 includes a medical condition diagnosis service140 that may receive speech data of a person and process the speech datato determine if a person has a medical condition. For example, medicalcondition diagnosis service 140 may process the speech data to compute ayes or no determination as to whether the person has the medicalcondition or to compute a score that indicates a probability or alikelihood that the person has the medical condition and/or a severityof the condition.

As used herein, a diagnosis relates to any determination as to whether aperson may have a medical condition or any determination as to apossible severity of the medical condition. A diagnosis may include anyform of an assessment, conclusion, opinion, or determination relating toa medical condition. In some instances, a diagnosis may be incorrect,and a person diagnosed with a medical condition may not actually havethe medical condition.

Medical condition diagnosis service 140 may receive the speech data of aperson using any appropriate techniques. For example, a person may speakto a mobile device 110 and mobile device 110 may record the speech andtransmit the recorded speech data to medical condition diagnosis service140 over network 130. Any appropriate techniques and any appropriatenetwork may be used for mobile device 110 to transmit the recordedspeech data to medical condition diagnosis service 140. For example, anapplication or “app” may be installed on mobile device 110 that uses aREST (representational state transfer) API (application programminginterface) call to transmit the speech data over the Internet or amobile telephone network. In another example, a medical provider mayhave a medical provider computer 120 that is used to record speech of aperson and transmit speech data to medical condition diagnosis service140.

In some implementations, medical condition diagnosis service 140 may beinstalled on mobile device 110 or medical provider computer 120 suchthat it is not necessary to transmit the speech data over a network. Theexample of FIG. 1 is not limiting, and any appropriate techniques may beused to transmit speech data for processing by a mathematical model.

The output of medical condition diagnosis service 140 may then be usedfor any appropriate purpose. For example, information may be presentedto the person who provided the speech data or to a medical professionalwho is treating the person.

FIG. 2 is an example system 200 for processing speech data with amathematical model to perform a medical diagnosis. In processing thespeech data, features may be computed from the speech data, and then thefeatures may be processed by the mathematical model. Any appropriatetype of features may be used.

The features may include acoustic features, where acoustic features areany features computed from the speech data that do not involve or dependon performing speech recognition on the speech data (e.g., the acousticfeatures do not use information about the words spoken in the speechdata). For example, acoustic features may include mel-frequency cepstralcoefficients, perceptual linear prediction features, jitter, or shimmer.

The features may include language features where language features arecomputed using the results of a speech recognition. For example,language features may include a speaking rate (e.g., the number ofvowels or syllables per second), a number of pause fillers (e.g., “ums”and “ahs”), the difficulty of words (e.g., less common words), or theparts of speech of words following pause fillers.

In FIG. 2, the speech data is processed by acoustic feature computationcomponent 210 and speech recognition component 220. Acoustic featurecomputation component 210 may compute acoustic features from the speechdata, such as any of the acoustic features described herein. Speechrecognition component 220 may perform automatic speech recognition onthe speech data using any appropriate techniques (e.g., Gaussian mixturemodels, acoustic modelling, language modelling, and neural networks).

Because speech recognition component 220 may use acoustic features inperforming speech recognition, some processing of these two componentsmay overlap and thus other configurations are possible. For example,acoustic feature component 210 may compute the acoustic features neededby speech recognition component 220, and speech recognition component220 may thus not need to compute any acoustic features.

Language feature computation component 230 may receive speechrecognition results from speech recognition component 220, and processthe speech recognition results to determine language features, such asany of the language features described herein. The speech recognitionresults may be in any appropriate format and include any appropriateinformation. For example, the speech recognition results may include aword lattice that includes multiple possible sequences of words,information about pause fillers, and the timings of words, syllables,vowels, pause fillers, or any other unit of speech.

Medical condition classifier 240 may process the acoustic features andthe language features with a mathematical model to output one or morediagnosis scores that indicate whether the person has the medicalcondition, such as a score indicating a probability or likelihood thatthe person has the medical condition and/or a score indicating aseverity of the medical condition. Medical condition classifier 240 mayuse any appropriate techniques, such as a classifier implemented with asupport vector machine or a neural network, such as a multi-layerperceptron.

The performance of medical condition classifier 240 may depend on thefeatures computed by acoustic feature computation component 210 andlanguage feature computation component 230. Further, a set of featuresthat performs well for one medical condition may not perform well foranother medical condition. For example, word difficulty may be animportant feature for diagnosing Alzheimer's disease but may not beuseful for determining if a person has a concussion. For anotherexample, features relating to the pronunciation of vowels, syllables, orwords may be important for Parkinson's disease but may be less importantfor other medical conditions. Accordingly, techniques are needed fordetermining a first set of features that performs well for a firstmedical condition, and this process may need to be repeated fordetermining a second set of features that performs well for a secondmedical condition.

In some implementations, medical condition classifier 240 may use otherfeatures, which may be referred to as non-speech features, in additionto acoustic features and language features. For example, features may beobtained or computed from demographic information of a person (e.g.,gender, age, or place of residence), information from a medical history(e.g., weight, recent blood pressure readings, or previous diagnoses),or any other appropriate information.

The selection of features for diagnosing a medical condition may be moreimportant in situations where an amount of training data for trainingthe mathematical model is relatively small. For example, for training amathematical model for diagnosing concussions, the needed training datamay include speech data of a number of individuals shortly after theyexperience a concussion. Such data may exist in small quantities andobtaining further examples of such data may take a significant period oftime.

Training mathematical models with a smaller amount of training data mayresult in overfitting where the mathematical model is adapted to thespecific training data but because of the small amount of training data,the model may not perform well on new data. For example, the model maybe able to detect all of the concussions in the training data, but mayhave a high error rate when processing production data of people who mayhave concussions.

One technique for preventing overfitting when training a mathematicalmodel is to reduce the number of features used to train the mathematicalmodel. The amount of training data needed to train a model withoutoverfitting increases as the number of features increases. Accordingly,using a smaller number of features allows models to be built with asmaller amount of training data.

Where it is needed to train a model with a smaller number of features,it becomes more important to select the features that will allow themodel to perform well. For example, when a large amount of training datais available, hundreds of features may be used to train the model and itis more likely that appropriate features have been used. Conversely,where a small amount of training data is available, only 10 or sofeatures may be used to train a model, and it is more important toselect the features that are most important for diagnosing the medicalcondition.

Now presented are examples of features that may be used to diagnose amedical condition.

Acoustic features may be computed using short-time segment features.When processing speech data, the duration of the speech data may vary.For example, some speech may be a second or two and other speech may beseveral minutes or more. For consistency in processing speech data, itmay be processed in short-time segments (sometimes referred to asframes). For example, each short-time segment may be 25 milliseconds,and segments may advance in increments of 10 milliseconds so that thereis a 15 millisecond overlap over two successive segments.

The following are non-limiting examples of short-time segment features:spectral features (such as mel-frequency cepstral coefficients orperceptual linear predictives); prosodic features (such as pitch,energy, or probability of voicing); voice quality features (such asjitter, jitter of jitter, shimmer, or harmonics-to-noise ratio); entropy(e.g., to capture how precisely an utterance is pronounced where entropymay be computed from the posteriors of an acoustic model that is trainedon natural speech data).

The short-time segment features may be combined to compute acousticfeatures for the speech. For example, a two-second speech sample mayproduce 200 short-time segment features for pitch that may be combinedto compute one or more acoustic features for pitch.

The short-time segment features may be combined to compute an acousticfeature for a speech sample using any appropriate techniques. In someimplementations, an acoustic feature may be computed using statistics ofthe short-time segment features (e.g., arithmetic mean, standarddeviation, skewness, kurtosis, first quartile, second quartile, thirdquartile, the second quartile minus the first quartile, the thirdquartile minus the first quartile, the third quartile minus the secondquartile, 0.01 percentile, 0.99 percentile, the 0.99 percentile minusthe 0.01 percentile, the percentage of short-time segments whose valuesare above a threshold (e.g., where the threshold is 75% of the rangeplus the minimum), the percentage of segments whose values are above athreshold (e.g., where the threshold is 90% of the range plus theminimum), the slope of a linear approximation of the values, the offsetof a linear approximation of the values, the linear error computed asthe difference of the linear approximation and the actual values, or thequadratic error computed as the difference of the linear approximationand the actual values. In some implementations, an acoustic feature maybe computed as an i-vector or identity vector of the short-time segmentfeatures. An identity vector may be computed using any appropriatetechniques, such as performing a matrix-to-vector conversion using afactor analysis technique and a Gaussian mixture model.

The following are non-limiting examples of language features. A speakingrate, such as by computing the duration of all spoken words divided bythe number of vowels or any other appropriate measure of speaking rate.A number of pause fillers that may indicate hesitation in speech, suchas (1) a number of pause fillers divided by the duration of spoken wordsor (2) a number of pause fillers divided by the number of spoken words.A measure of word difficulty or the use of less common words. Forexample, word difficulty may be computed using statistics of 1-gramprobabilities of the spoken words, such as by classifying wordsaccording to their frequency percentiles (e.g., 5%, 10%, 15%, 20%, 30%,or 40%). The parts of speech of words following pause fillers, such as(1) the counts of each part-of-speech class divided by the number ofspoken words or (2) the counts of each part-of-speech class divided bythe sum of all part-of-speech counts.

In some implementations, language features may include a determinationof whether a person answered a question correctly. For example, a personmay be asked what the current year is or who the President of the UnitedStates is. The person's speech may be processed to determine what theperson said in response to the question and to determine if the personanswered the question correctly.

To train a model for diagnosing a medical condition, a corpus oftraining data may be collected. The training corpus may include examplesof speech where the diagnosis of the person is known. For example, itmay be known that the person had no concussion, or a mild, moderate, orsevere concussion.

FIG. 3 illustrates an example of a training corpus that includes speechdata for training a model for diagnosing concussions. For example, therows of the table of FIG. 3 may correspond to database entries. In thisexample, each entry includes an identifier of a person, the knowndiagnosis of the person (e.g., no concussion or a mild, medium, orsevere concussion), an identifier of a prompt or question that waspresented to a person (e.g., “How are you today?”), and a filename of afile that contains the speech data. The training data may be stored inany appropriate format using any appropriate storage technology.

The training corpus may store a representation of a person's speechusing any appropriate format. For example, a speech data item of thetraining corpus may include digital samples of an audio signal receivedat a microphone or may include a processed version of the audio signal,such as mel-frequency cepstral coefficients.

A single training corpus may contain speech data relating to multiplemedical conditions, or a separate training corpus may be used for eachmedical condition (e.g., a first training corpus for concussions and asecond training corpus for Alzheimer's disease). A separate trainingcorpus may be used for storing speech data for people with no known ordiagnosed medical condition, as this training corpus may be used fortraining models for multiple medical conditions.

FIG. 4 illustrates an example of stored prompts that may be used todiagnose medical conditions. Each prompt may be presented to a person,either by a person (e.g., a medical professional) or a computer, toobtain speech of the person in response to the prompt. Each prompt mayhave a prompt identifier so that it may be cross referenced with theprompt identifier of the training corpus. The prompts of FIG. 4 may bestored using any appropriate storage technology, such as a database.

FIG. 5 is an exemplary system 500 that may be used to select featuresfor training a mathematical model for diagnosing a medical condition,and then using the selected features to train the mathematical model.System 500 may be used multiple times to select features for differentmedical conditions. For example, a first use of system 500 may selectfeatures for diagnosing concussions and a second use of system 500 mayselect features for diagnosing Alzheimer's disease.

FIG. 5 includes a training corpus 510 of speech data items for traininga mathematical model for diagnosing a medical condition. Training corpus510 may include any appropriate information, such as speech data ofmultiple people with and without the medical condition, a labelindicating whether or not person has the medical condition, and anyother information described herein.

Acoustic feature computation component 210, speech recognition component220, and language feature computation component 230 may be implementedas described above to compute acoustic and language features for thespeech data in the training corpus. Acoustic feature computationcomponent 210 and language feature computation component 230 may computea large number of features so that the best performing features may bedetermined. This may be in contrast to FIG. 2 where these components areused in a production system and thus these components may compute onlythe features that were previously selected.

Feature selection score computation component 520 may compute aselection score for each feature (which may be an acoustic feature, alanguage feature, or any other feature described herein). To compute aselection score for a feature, a pair of numbers may be created for eachspeech data item in the training corpus, where the first number of thepair is the value of the feature and the second number of the pair is anindicator of the medical condition diagnosis. The value for theindicator of the medical condition diagnosis may have two values (e.g.,0 if the person does not have the medical condition and 1 if the personhas the medical condition) or may have a larger number of values (e.g.,a real number between 0 and 1 or multiple integers indicating alikelihood or severity of the medical condition).

Accordingly, for each feature, a pair of numbers may be obtained foreach speech data item of the training corpus. FIGS. 6A and 6B illustratetwo conceptual plots of the pairs of numbers for a first feature and asecond feature. For FIG. 6A, there does not appear to be a pattern orcorrelation between the values of the first feature and thecorresponding diagnosis values, but for FIG. 6B, there does appear to bea pattern or correlation between the values of the second feature andthe diagnosis values. Accordingly, one may conclude that the secondfeature is likely a useful feature for determining whether a person hasthe medical condition and that the first feature is not.

Feature selection score computation component 520 may compute aselection score for a feature using the pairs of feature values anddiagnosis values. Feature selection score computation component 520 maycompute any appropriate score that indicates a pattern or correlationbetween the feature values and the diagnosis values. For example,feature selection score computation component 520 may compute a Randindex, an adjusted Rand index, mutual information, adjusted mutualinformation, a Pearson correlation, an absolute Pearson correlation, aSpearman correlation, or an absolute Spearman correlation.

The selection score may indicate the usefulness of the feature indetecting a medical condition. For example, a high selection score mayindicate that a feature should be used in training the mathematicalmodel, and a low selection score may indicate that the feature shouldnot be used in training the mathematical model.

Feature stability determination component 530 may determine if a feature(which may be an acoustic feature, a language feature, or any otherfeature described herein) is stable or unstable. To make a stabilitydetermination, the speech data items may be divided into multiplegroups, which may be referred to as folds. For example, the speech dataitems may be divided into five folds. In some implementations, thespeech data items may be divided into folds such that each fold has anapproximately equal number of speech data items for different gendersand age groups.

The statistics of each fold may be compared to statistics of the otherfolds. For example, for a first fold, the median (or mean or any otherstatistic relating to the center or middle of a distribution) featurevalue (denoted as M₁) may be determined. Statistics may also be computedfor the combination of the other folds. For example, for the combinationof the other folds, the median of the feature values (denoted as M_(o))and a statistic measuring of variability of the feature values (denotedas V_(o)), such as interquartile range, variance, or standard deviation,may be computed. The feature may be determined to be unstable if themedian of the first fold differs too greatly from the median of thesecond fold. For example, the feature may be determined to be unstableif

$M_{1} < {M_{o} - {C\frac{V_{o}}{2}\mspace{14mu} {or}\mspace{14mu} M_{1}}} > {M_{o} + {C\frac{V_{o}}{2}}}$

where C is a scaling factor. The process may then be repeated for eachof the other folds. For example, the median of a second fold may becompared with median and variability of the other folds as describedabove.

In some implementations, if, after comparing each fold to the otherfolds, the median of each fold is not too far from the median of theother folds, then the feature may be determined to be stable.Conversely, if the median of any fold is too far from the median of theother folds, then the feature may be determined to be unstable.

In some implementations, feature stability determination component 530may output a boolean value for each feature to indicate whether thefeature is stable or not. In some implementations, stabilitydetermination component 530 may output a stability score for eachfeature. For example, a stability score may be computed as largestdistance between the median of a fold and the other folds (e.g., aMahalanobis distance).

Feature selection component 540 may receive the selection scores fromfeature selection score computation component 520 and the stabilitydeterminations from feature stability determination component 530 andselect a subset of features to be used to train the mathematical model.Feature selection component 540 may select a number of features havingthe highest selection scores that are also sufficiently stable.

In some implementations, the number of features to be selected (or amaximum number of features to be selected) may be set ahead of time. Forexample, a number N may be determined based on the amount of trainingdata, and N features may be selected. The selected features may bedetermined by removing unstable features (e.g., features determined tobe unstable or features with a stability score below a threshold) andthen selecting the N features with the highest selection scores.

In some implementations, the number of features to be selected may bebased on the selection scores and stability determinations. For example,the selected features may be determined by removing unstable features,and then selecting all features with a selection score above athreshold.

In some implementations, the selection scores and stability scores maybe combined when selecting features. For example, for each feature acombined score may be computed (such as by adding or multiplying theselection score and the stability score for the feature) and featuresmay be selected using the combined score.

Model training component 550 may then train a mathematical model usingthe selected features. For example, model training component 550 mayiterate over the speech data items of the training corpus, obtain theselected features for the speech data items, and then train themathematical model using the selected features. In some implementations,dimension reduction techniques, such as principal components analysis orlinear discriminant analysis, may be applied to the selected features aspart of the model training. Any appropriate mathematical model may betrained, such as any of the mathematical models described herein.

In some implementations, other techniques, such as wrapper methods, maybe used for feature selection or may be used in combination with thefeature selection techniques presented above. Wrapper methods may selecta set of features, train a mathematical model using the selected set offeatures, and then evaluate the performance of the set of features usingthe trained model. Where the number of possible features is relativelysmall and/or training time is relatively short, all possible sets offeatures may be evaluated and the best performing set may be selected.Where the number of possible features is relatively large and/or thetraining time is a significant factor, optimization techniques may beused to iteratively find a set of features that performs well. In someimplementations, a set of features may be selected using system 500, andthen a subset of these features may be selected using wrapper methods asthe final set of features.

FIG. 7 is a flowchart of an example implementation of selecting featuresfor training a mathematical model for diagnosing a medical condition. InFIG. 7 and other flowcharts herein, the ordering of the steps isexemplary and other orders are possible, not all steps are required,steps may be combined (in whole or part) or sub-divided and, in someimplementations, some steps may be omitted or other steps may be added.The methods described by any flowcharts described herein may beimplemented, for example, by any of the computers or systems describedherein.

At step 710, a training corpus of speech data items is obtained. Thetraining corpus may include a representation of an audio signal of aperson's speech, an indication of a medical diagnosis of the person fromwhom the speech was obtained, and any other appropriate information,such as any of the information described herein.

At step 720, speech recognition results are obtained for each speechdata item of the training corpus. The speech recognition results mayhave been computed in advance and stored with the training corpus orstored in another location. The speech recognition results may includeany appropriate information, such as a transcript, a list of highestscoring transcripts (e.g., an N-best list), a lattice of possibletranscriptions, and timing information, such as the start and end timeof words, pause fillers, or other speech units.

At step 730, acoustic features are computed for each speech data item ofthe training corpus. Acoustic features may include any features that arecomputed without using speech recognition results of a speech data item,such as any of the acoustic features described herein. Acoustic featuresmay include or be computed from data used in the speech recognitionprocess (e.g., mel-frequency cepstral coefficients or perceptual linearpredictors), but acoustic features do not use speech recognitionresults, such as information about the words or pause fillers present ina speech data item.

At step 740, language features are computed for each speech data item ofthe training corpus. Language features may include any features that arecomputed using speech recognition results, such as any of the languagefeatures described herein.

At step 750, a feature selection score is computed for each acousticfeature and each language feature. To compute a feature selection scorefor the feature, the value of the feature for each speech data item inthe training corpus may be used along with other information, such as aknown diagnosis value corresponding to the speech data item. The featureselection score may be computed using any of the techniques describedherein, such as by computing an absolute Pearson correlation. In someimplementations, feature selection scores may be computed for otherfeatures as well, such as features relating to demographic informationof a person.

At step 760, a plurality of features is selected using the featureselection scores. For example, a number of features having the highestselection scores may be selected. In some implementations, a stabilitydetermination may be computed for each feature and the plurality offeatures may be selected using both the feature selection scores and thestability determinations, such as by using any of the techniquesdescribed herein.

At step 770, a mathematical model is trained using the selectedfeatures. Any appropriate mathematical model may be trained, such as aneural network or a support vector machine. After the mathematical modelhas been trained, it may be deployed in a production system, such assystem 100 of FIG. 1 to perform diagnosis of medical conditions.

The steps of FIG. 7 may be performed in a variety of manners. Forexample, in some implementations, steps 730, and 740 may be performed ina loop that loops over each of the speech data items in the trainingcorpus. For a first iteration, acoustic and language features may becomputed for a first speech data item, for a second iteration, acousticand language features may be computed for a second speech data item, andso forth.

When using a deployed model for diagnosing a medical condition, theperson being diagnosed may be presented with a sequence of prompts orquestions to obtain speech from the person. Any appropriate prompts maybe used, such as any of the prompts of FIG. 4. After the features havebeen selected, as described above, prompts may be selected so that theselected prompts provide useful information about the selected features.

For example, suppose that a selected feature is pitch. While pitch hasbeen determined to be a useful feature for diagnosing a medicalcondition, some prompts may be better than others in obtaining a usefulpitch feature. Very short utterances (e.g., yes/no answers) may notprovide sufficient data to accurately compute pitch and thus promptsthat generate longer responses may be more useful in obtaininginformation about pitch.

For another example, suppose that a selected feature is word difficulty.While word difficulty has been determined to be a useful feature fordiagnosing a medical condition, some prompts may be better than othersin obtaining a useful word difficulty feature. Prompts that ask a userto read a presented passage will generally result in speech of the wordsin the passage, and thus the word difficulty feature would have the samevalue each time this prompt is presented, and thus this prompt would notbe useful in obtaining information about word difficulty. By contrast,open ended questions, such as “Tell me about your day?”, may result ingreater variability of vocabulary in responses and thus may provide moreuseful information about word difficulty.

Selecting a set of prompts may also improve the performance of a systemfor diagnosing medical conditions and provide a better experience forthe person being evaluated. By using the same set of prompts for eachperson being evaluated, the system for diagnosing medical conditions mayprovide more accurate results, since the data collected from multiplepeople may be more comparable than if different prompts were used witheach person. Further, using a defined set of prompts, allows theevaluation of a person to be more predictable and of a desired durationthat is appropriate for the evaluation of the medical condition. Forexample, for evaluating whether a person has Alzheimer's disease, it maybe acceptable to use more prompts to collect a larger amount of data,but for evaluating whether a person has a concussion during a sportingevent, it may be necessary to use a smaller number of prompts to obtaina result more quickly.

In some implementations, prompts may be selected by computing promptselection scores. A training corpus may have multiple or even manyspeech data items for a single prompt. For example, the training corpusmay include examples of the prompt used with different people or thesame prompt may be used with the same person multiple times.

FIG. 8 is a flowchart of an example implementation of selecting promptsfor use with a deployed model for diagnosing a medical condition.

Steps 810 to 840 may be performed for each prompt (or a subset of theprompts) in the training corpus to compute a prompt selection score foreach prompt.

At step 810 a prompt is obtained, and at step 820 speech data itemscorresponding to the prompt are obtained from the training corpus.

At step 830, a medical diagnosis score is computed for each speech dataitem corresponding to the prompt. For example, a medical diagnosis scorefor a speech data item may be a number output by a mathematical model(e.g., the mathematical model trained in FIG. 7) indicating a likelihoodthat a person has the medical condition and/or a severity of the medicalcondition.

At step 840, a prompt selection score is computed for the prompt usingthe computed medical diagnosis scores. The computation of a promptselection score may be similar to the computation of a feature selectionscore, as described above. For each speech data item corresponding tothe prompt, a pair of numbers may be obtained. For each pair, the firstnumber of the pair may be the computed medical diagnosis score computedfrom the speech data item, and the second number of the pair may be aknown medical condition diagnosis of the person (e.g., the person isknown to have the medical condition or a severity of the medicalcondition). Plotting these pairs of numbers may result in a plot similarto FIG. 6A or FIG. 6B, and depending on the prompt there may or may notbe a pattern or correlation in the pairs of numbers.

A prompt selection score for a prompt may include any score thatindicates a pattern or correlation between the computed medicaldiagnosis scores and the known medical condition diagnoses. For example,a prompt selection score may include a Rand index, an adjusted Randindex, mutual information, adjusted mutual information, a Pearsoncorrelation, an absolute Pearson correlation, a Spearman correlation, oran absolute Spearman correlation.

At step 850 it is determined if other prompts remain to be processed. Ifprompts remain to be processed, then processing may proceed to step 810to process additional prompts. If all prompts have been processed, thenprocessing may proceed to step 860.

At step 860, a plurality of prompts is selected using the promptselection scores. For example, a number of prompts having the highestprompt selection scores may be selected. In some implementations, astability determination may be computed for each prompt and theplurality of prompts may be selected using both the prompt selectionscores and the prompt stability determinations, such as by using any ofthe techniques described herein.

At step 870, the selected prompts are used with a deployed medicalcondition diagnosis service. For example, when diagnosing a person, theselected prompts may be presented to a person to obtain speech of theperson in response to each of the prompts.

In some implementations, other techniques, such as wrapper methods, maybe used for prompt selection or may be used in combination with theprompt selection techniques presented above. In some implementations, aset of prompts may be selected using the process of FIG. 8, and then asubset of these prompts may be selected using wrapper methods as thefinal set of features.

In some implementations, a person involved with creating the medicalcondition diagnosis service may assist in the selection of prompts. Theperson may use his knowledge or experience to select prompts based onthe selected features. For example, where a selected feature is worddifficulty, the person may review the prompts and select prompts thatare more likely to provide useful information relating to worddifficulty. The person may select one or more prompts that are likely toprovide useful information for each of the selected features.

In some implementations, the person may review the prompts selected bythe process of FIG. 8, and add or remove prompts to improve theperformance of a medical condition diagnosis system. For example, twoprompts may each provide useful information about word difficulty, butthe information provided by the two prompts may be largely redundant,and using both prompts may not provide significant benefit over usingjust one of them.

In some implementations, a second mathematical model may be trainedafter prompt selection that is adapted to the selected prompts. Themathematical model trained in FIG. 7 may process a single utterance (inresponse to a prompt) to generate a medical diagnosis score. Where theprocess of performing a diagnosis comprises processing multipleutterances corresponding to multiple prompts, then each of theutterances may be processed by the mathematical model of FIG. 7 togenerate multiple medical diagnosis scores. To determine an overallmedical diagnosis, the multiple medical diagnosis scores may need to becombined in some way. Accordingly, the mathematical model trained inFIG. 7 may not be adapted to a selected set of prompts.

When the selected prompts are used in a session to diagnose a person,each of the prompts may be presented to the person to obtain anutterance corresponding to each of the prompts. Instead of processingthe utterances separately, the utterances may be processedsimultaneously by the model to generate a medical diagnosis score.Accordingly, a model may be adapted to the selected prompts because itis trained to simultaneously process utterances corresponding to each ofthe selected prompts.

FIG. 9 is a flowchart of an example implementation training amathematical model that is adapted to a set of selected prompts. At step910, a first mathematical model is obtained, such as by using theprocess of FIG. 7. At step 920, a plurality of prompts is selected usingthe first mathematical model, such as by the process of FIG. 8.

At step 930, a second mathematical model is trained that simultaneouslyprocesses multiple speech data items corresponding to the plurality ofselected prompts to generate a medical diagnosis score. When trainingthe second mathematical model, a training corpus may be used thatincludes sessions with speech data items corresponding to each of theplurality of selected prompts. When training the mathematical model, theinput to the mathematical model may be fixed to the speech data itemsfrom the session and corresponding to each of the selected prompts. Theoutput of the mathematical model may be fixed to a known medicaldiagnosis. The parameters of the model may then be trained to optimallyprocess the speech data item simultaneously to generate a medicaldiagnosis score. Any appropriate training techniques may be used, suchas stochastic gradient descent.

The second mathematical model may then be deployed as part of a medicalcondition diagnosis service, such as the service of FIG. 1. The secondmathematical model may provide better performance than the firstmathematical model because it has been trained to process the utterancessimultaneously rather than individual and thus the training may bebetter able to combine the information from all the of utterances togenerate the medical condition diagnosis score.

FIG. 10 illustrates components of one implementation of a computingdevice 1000 for implementing any of the techniques described above. InFIG. 10, the components are shown as being on a single computing device,but the components may be distributed among multiple computing devices,such as a system of computing devices, including, for example, anend-user computing device (e.g., a smart phone or a tablet) and/or aserver computing device (e.g., cloud computing).

Computing device 1000 may include any components typical of a computingdevice, such as volatile or nonvolatile memory 1010, one or moreprocessors 1011, and one or more network interfaces 1012. Computingdevice 1000 may also include any input and output components, such asdisplays, keyboards, and touch screens. Computing device 1000 may alsoinclude a variety of components or modules providing specificfunctionality, and these components or modules may be implemented insoftware, hardware, or a combination thereof. Below, several examples ofcomponents are described for one example implementation, and otherimplementations may include additional components or exclude some of thecomponents described below.

Computing device 1000 may have an acoustic feature computation component1021 that may compute acoustic features for a speech data item asdescribed above. Computing device 1000 may have a language featurecomputation component 1022 that may compute language features for aspeech data item as described above. Computing device 1000 may have aspeech recognition component 1023 that may generate speech recognitionresults for a speech data item as described above. Computing device 1000may have a feature selection score computation component 1031 that maycompute selection scores for features as described above. Computingdevice 1000 may have a feature stability score computation component1032 that may make stability determinations or compute stability scoresas described above. Computing device 1000 may have a feature selectioncomponent 1033 that may select features using selection scores and/orstability determinations as described above. Computing device 1000 mayhave a prompt selection score computation component 1041 that maycompute selection scores for prompts as described above. Computingdevice 1000 may have a prompt stability score computation component 1042that may make stability determinations or compute stability scores asdescribed above. Computing device 1000 may have a prompt selectioncomponent 1043 that may select prompts using selection scores and/orstability determinations as described above. Computing device 1000 mayhave a model training component 1050 that may train mathematical modelsas described above. Computing device 1000 may have a medical conditiondiagnosis component 1060 that may process speech data items to determinea medical diagnosis score as described above.

Computing device 1000 may include or have access to various data stores,such as training corpus data store 1070. Data stores may use any knownstorage technology such as files, relational or non-relationaldatabases, or any non-transitory computer-readable media.

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software, program codes,and/or instructions on a processor. “Processor” as used herein is meantto include at least one processor and unless context clearly indicatesotherwise, the plural and the singular should be understood to beinterchangeable. Any aspects of the present disclosure may beimplemented as a method on the machine, as a system or apparatus as partof or in relation to the machine, or as a computer program productembodied in a computer readable medium executing on one or more of themachines. The processor may be part of a server, client, networkinfrastructure, mobile computing platform, stationary computingplatform, or other computing platform. A processor may be any kind ofcomputational or processing device capable of executing programinstructions, codes, binary instructions and the like. The processor maybe or include a signal processor, digital processor, embedded processor,microprocessor or any variant such as a co-processor (math co-processor,graphic co-processor, communication co-processor and the like) and thelike that may directly or indirectly facilitate execution of programcode or program instructions stored thereon. In addition, the processormay enable execution of multiple programs, threads, and codes. Thethreads may be executed simultaneously to enhance the performance of theprocessor and to facilitate simultaneous operations of the application.By way of implementation, methods, program codes, program instructionsand the like described herein may be implemented in one or more thread.The thread may spawn other threads that may have assigned prioritiesassociated with them; the processor may execute these threads based onpriority or any other order based on instructions provided in theprogram code. The processor may include memory that stores methods,codes, instructions and programs as described herein and elsewhere. Theprocessor may access a storage medium through an interface that maystore methods, codes, and instructions as described herein andelsewhere. The storage medium associated with the processor for storingmethods, programs, codes, program instructions or other type ofinstructions capable of being executed by the computing or processingdevice may include but may not be limited to one or more of a CD-ROM,DVD, memory, hard disk, flash drive, RAM, ROM, cache and the like.

A processor may include one or more cores that may enhance speed andperformance of a multiprocessor. In embodiments, the process may be adual core processor, quad core processors, other chip-levelmultiprocessor and the like that combine two or more independent cores(called a die).

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software on a server,client, firewall, gateway, hub, router, or other such computer and/ornetworking hardware. The software program may be associated with aserver that may include a file server, print server, domain server,internet server, intranet server and other variants such as secondaryserver, host server, distributed server and the like. The server mayinclude one or more of memories, processors, computer readable media,storage media, ports (physical and virtual), communication devices, andinterfaces capable of accessing other servers, clients, machines, anddevices through a wired or a wireless medium, and the like. The methods,programs, or codes as described herein and elsewhere may be executed bythe server. In addition, other devices required for execution of methodsas described in this application may be considered as a part of theinfrastructure associated with the server.

The server may provide an interface to other devices including, withoutlimitation, clients, other servers, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more locations without deviating from the scope ofthe disclosure. In addition, any of the devices attached to the serverthrough an interface may include at least one storage medium capable ofstoring methods, programs, code and/or instructions. A centralrepository may provide program instructions to be executed on differentdevices. In this implementation, the remote repository may act as astorage medium for program code, instructions, and programs.

The software program may be associated with a client that may include afile client, print client, domain client, internet client, intranetclient and other variants such as secondary client, host client,distributed client and the like. The client may include one or more ofmemories, processors, computer readable media, storage media, ports(physical and virtual), communication devices, and interfaces capable ofaccessing other clients, servers, machines, and devices through a wiredor a wireless medium, and the like. The methods, programs, or codes asdescribed herein and elsewhere may be executed by the client. Inaddition, other devices required for execution of methods as describedin this application may be considered as a part of the infrastructureassociated with the client.

The client may provide an interface to other devices including, withoutlimitation, servers, other clients, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more locations without deviating from the scope ofthe disclosure. In addition, any of the devices attached to the clientthrough an interface may include at least one storage medium capable ofstoring methods, programs, applications, code and/or instructions. Acentral repository may provide program instructions to be executed ondifferent devices. In this implementation, the remote repository may actas a storage medium for program code, instructions, and programs.

The methods and systems described herein may be deployed in part or inwhole through network infrastructures. The network infrastructure mayinclude elements such as computing devices, servers, routers, hubs,firewalls, clients, personal computers, communication devices, routingdevices and other active and passive devices, modules and/or componentsas known in the art. The computing and/or non-computing device(s)associated with the network infrastructure may include, apart from othercomponents, a storage medium such as flash memory, buffer, stack, RAM,ROM and the like. The processes, methods, program codes, instructionsdescribed herein and elsewhere may be executed by one or more of thenetwork infrastructural elements.

The methods, program codes, and instructions described herein andelsewhere may be implemented on a cellular network having multiplecells. The cellular network may either be frequency division multipleaccess (FDMA) network or code division multiple access (CDMA) network.The cellular network may include mobile devices, cell sites, basestations, repeaters, antennas, towers, and the like. The cell networkmay be a GSM, GPRS, 3G, EVDO, mesh, or other networks types.

The methods, programs codes, and instructions described herein andelsewhere may be implemented on or through mobile devices. The mobiledevices may include navigation devices, cell phones, mobile phones,mobile personal digital assistants, laptops, palmtops, netbooks, pagers,electronic books readers, music players and the like. These devices mayinclude, apart from other components, a storage medium such as a flashmemory, buffer, RAM, ROM and one or more computing devices. Thecomputing devices associated with mobile devices may be enabled toexecute program codes, methods, and instructions stored thereon.Alternatively, the mobile devices may be configured to executeinstructions in collaboration with other devices. The mobile devices maycommunicate with base stations interfaced with servers and configured toexecute program codes. The mobile devices may communicate on apeer-to-peer network, mesh network, or other communications network. Theprogram code may be stored on the storage medium associated with theserver and executed by a computing device embedded within the server.The base station may include a computing device and a storage medium.The storage device may store program codes and instructions executed bythe computing devices associated with the base station.

The computer software, program codes, and/or instructions may be storedand/or accessed on machine readable media that may include: computercomponents, devices, and recording media that retain digital data usedfor computing for some interval of time; semiconductor storage known asrandom access memory (RAM); mass storage typically for more permanentstorage, such as optical discs, forms of magnetic storage like harddisks, tapes, drums, cards and other types; processor registers, cachememory, volatile memory, non-volatile memory; optical storage such asCD, DVD; removable media such as flash memory (e.g. USB sticks or keys),floppy disks, magnetic tape, paper tape, punch cards, standalone RAMdisks, Zip drives, removable mass storage, off-line, and the like; othercomputer memory such as dynamic memory, static memory, read/writestorage, mutable storage, read only, random access, sequential access,location addressable, file addressable, content addressable, networkattached storage, storage area network, bar codes, magnetic ink, and thelike.

The methods and systems described herein may transform physical and/oror intangible items from one state to another. The methods and systemsdescribed herein may also transform data representing physical and/orintangible items from one state to another.

The elements described and depicted herein, including in flow charts andblock diagrams throughout the figures, imply logical boundaries betweenthe elements. However, according to software or hardware engineeringpractices, the depicted elements and the functions thereof may beimplemented on machines through computer executable media having aprocessor capable of executing program instructions stored thereon as amonolithic software structure, as standalone software modules, or asmodules that employ external routines, code, services, and so forth, orany combination of these, and all such implementations may be within thescope of the present disclosure. Examples of such machines may include,but may not be limited to, personal digital assistants, laptops,personal computers, mobile phones, other handheld computing devices,medical equipment, wired or wireless communication devices, transducers,chips, calculators, satellites, tablet PCs, electronic books, gadgets,electronic devices, devices having artificial intelligence, computingdevices, networking equipment, servers, routers and the like.Furthermore, the elements depicted in the flow chart and block diagramsor any other logical component may be implemented on a machine capableof executing program instructions. Thus, while the foregoing drawingsand descriptions set forth functional aspects of the disclosed systems,no particular arrangement of software for implementing these functionalaspects should be inferred from these descriptions unless explicitlystated or otherwise clear from the context. Similarly, it will beappreciated that the various steps identified and described above may bevaried, and that the order of steps may be adapted to particularapplications of the techniques disclosed herein. All such variations andmodifications are intended to fall within the scope of this disclosure.As such, the depiction and/or description of an order for various stepsshould not be understood to require a particular order of execution forthose steps, unless required by a particular application, or explicitlystated or otherwise clear from the context.

The methods and/or processes described above, and steps thereof, may berealized in hardware, software or any combination of hardware andsoftware suitable for a particular application. The hardware may includea general-purpose computer and/or dedicated computing device or specificcomputing device or particular aspect or component of a specificcomputing device. The processes may be realized in one or moremicroprocessors, microcontrollers, embedded microcontrollers,programmable digital signal processors or other programmable device,along with internal and/or external memory. The processes may also, orinstead, be embodied in an application specific integrated circuit, aprogrammable gate array, programmable array logic, or any other deviceor combination of devices that may be configured to process electronicsignals. It will further be appreciated that one or more of theprocesses may be realized as a computer executable code capable of beingexecuted on a machine-readable medium.

The computer executable code may be created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software, or any other machinecapable of executing program instructions.

Thus, in one aspect, each method described above and combinationsthereof may be embodied in computer executable code that, when executingon one or more computing devices, performs the steps thereof. In anotheraspect, the methods may be embodied in systems that perform the stepsthereof, and may be distributed across devices in a number of ways, orall of the functionality may be integrated into a dedicated, standalonedevice or other hardware. In another aspect, the means for performingthe steps associated with the processes described above may include anyof the hardware and/or software described above. All such permutationsand combinations are intended to fall within the scope of the presentdisclosure.

While the invention has been disclosed in connection with the preferredembodiments shown and described in detail, various modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present invention isnot to be limited by the foregoing examples, but is to be understood inthe broadest sense allowable by law.

All documents referenced herein are hereby incorporated by reference.

What is claimed is:
 1. A system for training a mathematical model fordetecting a medical condition, the system comprising at least onecomputer configured to: obtain a training corpus comprising speech dataitems, wherein each speech data item is labelled with a diagnosis value;compute a plurality of features for each speech data item in thetraining corpus; compute a feature selection score for each feature ofthe plurality of features, wherein: the feature selection score for afeature indicates a usefulness of the feature for detecting the medicalcondition, and the feature selection score is computed using, for eachspeech data item, a value of the feature and the diagnosis valuecorresponding to the speech data item; select a subset of the pluralityof features using the feature selection scores; train the mathematicalmodel for detecting the medical condition using the subset of theplurality of features for each speech data item of the training corpus;deploy a computer program product or computer service for detecting themedical condition using the mathematical model; present, by the computerprogram product or computer service, a prompt to a person; receive, bythe computer program product or computer service, a speech data itemcorresponding to speech of a person in response to the prompt; compute amedical diagnosis score by processing the received speech data itemusing the mathematical model; and display, by the computer programproduct or computer service, one or more of the medical diagnosis scoreor a medical diagnosis based on the medical diagnosis score.
 2. Thesystem of claim 1, wherein the at least one computer is configured to:obtain speech recognition results for each speech data item of thetraining corpus, wherein the speech recognition results for a speechdata item comprise a transcription of the speech data item; compute alanguage feature for each speech data item in the training corpus byprocessing the speech recognition results; and wherein the plurality offeatures comprise the language feature.
 3. The system of claim 1,wherein each speech data item of the training corpus corresponds to aprompt of a plurality of prompts, the plurality of prompts comprisingthe presented prompt, and wherein the at least one computer isconfigured to: compute a medical diagnosis score for each speech dataitem of the training corpus by processing the speech data items with themathematical model; compute a prompt selection score for each prompt ofthe plurality of prompts using the medical diagnosis scores; select asubset of prompts from the plurality of prompts using the promptselection scores, the subset of prompts comprising the presented prompt;deploy the computer program product or computer service for detectingthe medical condition using the mathematical model and the subset ofprompts; receive a speech data item corresponding to speech of theperson for each prompt of the subset of prompts; and compute the medicaldiagnosis score for the person by processing the received speech dataitems using the mathematical model.
 4. The system of claim 1, whereinthe at least one computer is configured to: compute an acoustic featurefor each speech data item in the training corpus, wherein the acousticfeature is computed from the speech data item and wherein computation ofthe acoustic feature does not use speech recognition results of thespeech data item; and wherein the plurality of features comprise theacoustic feature.
 5. The system of claim 4, wherein the plurality offeatures comprise a language feature computed from speech recognitionresults of a speech data item.
 6. The system of claim 1, wherein themathematical model comprises a neural network or a support vectormachine.
 7. The system of claim 1, wherein the plurality of featurescomprises at least one of spectral features, prosodic features, or voicequality features.
 8. A computer-implemented method for training amathematical model for detecting a medical condition, the methodcomprising: obtaining a training corpus comprising speech data items,wherein each speech data item is labelled with a diagnosis value;computing a plurality of features for each speech data item in thetraining corpus; computing a feature selection score for each feature ofthe plurality of features, wherein: the feature selection score for afeature indicates a usefulness of the feature for detecting the medicalcondition, and the feature selection score is computed using, for eachspeech data item, a value of the feature and the diagnosis valuecorresponding to the speech data item; selecting a subset of theplurality of features using the feature selection scores; training themathematical model for detecting the medical condition using the subsetof the plurality of features for each speech data item of the trainingcorpus; deploying a computer program product or computer service fordetecting the medical condition using the mathematical model;presenting, by the computer program product or computer service, aprompt to a person; receiving, by the computer program product orcomputer service, a speech data item corresponding to speech of a personin response to the prompt; computing a medical diagnosis score byprocessing the received speech data item using the mathematical model;and displaying, by the computer program product or computer service, oneor more of the medical diagnosis score or a medical diagnosis based onthe medical diagnosis score.
 9. The computer-implemented method of claim8, wherein the medical condition is a concussion or Alzheimer's disease.10. The computer-implemented method of claim 8, wherein the plurality offeatures comprises one or more of a number of pause fillers over aperiod of time, a number of pause fillers over a number of words, worddifficulty, or speaking rate.
 11. The computer-implemented method ofclaim 8, wherein computing a feature selection score for a featurecomprises generating a pair of numbers for each speech data item of thetraining corpus, and wherein a first number of the pair corresponds to afeature value and a second number of the pair corresponds to a diagnosisvalue.
 12. The computer-implemented method of claim 8, comprising:dividing the training corpus into a plurality of folds; and computing astatistic for each feature and each fold of the plurality of folds. 13.The computer-implemented method of claim 12, comprising: computing astability determination for each feature of the plurality of featuresusing the statistics for each feature and each fold of the plurality offolds; and selecting the subset of the plurality of features using thestability determinations.
 14. The computer-implemented method of claim8, comprising: selecting a plurality of prompts using the mathematicalmodel; and training a second mathematical model using the selectedplurality of prompts and the speech data items of the training corpus.15. One or more non-transitory computer-readable media comprisingcomputer executable instructions that, when executed, cause at least oneprocessor to perform actions comprising: obtaining a training corpuscomprising speech data items, wherein each speech data item is labelledwith a diagnosis value; obtaining a plurality of features for eachspeech data item in the training corpus; computing a feature selectionscore for each feature of the plurality of features, wherein: thefeature selection score for a feature indicates a usefulness of thefeature for detecting a medical condition, and the feature selectionscore is computed using, for each speech data item, a value of thefeature and the diagnosis value corresponding to the speech data item;selecting a subset of the plurality of features using the featureselection scores; training a mathematical model for detecting themedical condition using the subset of the plurality of features for eachspeech data item of the training corpus; deploying a computer programproduct or computer service for detecting the medical condition usingthe mathematical model; presenting, by the computer program product orcomputer service, a prompt to a person; receiving, by the computerprogram product or computer service, a speech data item corresponding tospeech of a person in response to the prompt; computing a medicaldiagnosis score by processing the received speech data item using themathematical model; and displaying, by the computer program product orcomputer service, one or more of the medical diagnosis score or amedical diagnosis based on the medical diagnosis score.
 16. The one ormore non-transitory computer-readable media of claim 15, whereincomputing a first feature of the plurality of features comprises:computing a value for each short-time segment of an audio signal toobtain a plurality of values; and computing the first feature using theplurality of values.
 17. The one or more non-transitorycomputer-readable media of claim 15, wherein the feature selection scorecomprises an adjusted Rand index, adjusted mutual information, anabsolute Pearson correlation, or an absolute Spearman correlation. 18.The one or more non-transitory computer-readable media of claim 15,wherein the actions comprise: computing a stability determination foreach feature of the plurality of features; and selecting the pluralityof features using the stability determinations.
 19. The one or morenon-transitory computer-readable media of claim 15, wherein each speechdata item of the training corpus corresponds to a prompt of a pluralityof prompts, the plurality of prompts comprising the presented prompt,and wherein the actions comprise: computing a medical diagnosis scorefor each speech data item of the training corpus by processing thespeech data items with the mathematical model; computing a promptselection score for each prompt of the plurality of prompts using themedical diagnosis scores; selecting a subset of prompts from theplurality of prompts using the prompt selection scores, the subset ofprompts comprising the presented prompt; and deploying the computerprogram product or computer service for detecting the medical conditionusing the mathematical model and the subset of prompts.
 20. The one ormore non-transitory computer-readable media of claim 15, wherein theplurality of features comprise a non-speech feature.